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Abstract 

This paper presents on-going research in robotic inspec- 
tion of space platforms. Three main areas of inves- 
tigation are discussed; machine-vision inspection tech- 
niques, an integrated sensor end-effector, and an orbital 
environment laboratory simulation. Machine- vision in- : 
spection utilizes automatic comparison of new and ref- 
erence images to detect on-orbit induced damage such 
as micro-meteorite impacts. The cameras and light- 
ing used for this inspection are housed in a multi- 
sensor end-effector , which also contains a suite of sen- 
sors for detection of temperature , gas leaks , proximity , 
and forces. To fully test all of these sensors, a realis- 
tic space platform mock-up has been created, complete 
with visual, temperature, and gas anomalies. Further, 
changing orbital lighting conditions are effectively mim- 
icked by a robotic solar simulator. In the paper, each 
of these technology components will be discussed, and 
experimental results are provided. 

1 Introduction 

Later this decade, NASA will place in orbit around 
Earth the Space Station Freedom (SSF), which will be 
used as a science station and home for astronauts for 30 
years. Soon after its initial design, engineering reviews 
revealed that simple inspection and maintenance of the 
station would consume more time than the astronauts 
would have available [2]. This was reinforced by results 
of the Long Duration Exposure Facility (LDEF), which 
showed large amounts of damage from micro-meteorite 
impacts and atomic oxygen degradation while in orbit 
for five years [8]. For these reasons, NASA sponsored 
The Remote Surface Inspection Task (RSI), a five year 
technology demonstration task at the Jet Propulsion 
Laboratory, California Institute of Technology (JPL). 
This project has developed and systematically investi- 
gated methods for telerobotic inspection of SSF [4]. 
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The inspection system which has been built for 
this research is comprised of three main subsystems: 
robot manipulator control, graphical user interfacing, 
and teleoperated/automated multi-sensor inspection. 
The robot manipulator subsystem is comprised of a 
Robotics Research K1207 arm mounted on a translating 
platform, and controlled by a real-time system employ- 
ing Configuration Control [9]. The graphical user inter- 
face subsystem resides on an SGI workstation and pro- 
vides user-friendly interfaces to the manipulator control 
and the inspection data [6]. The multi-sensor inspection 
subsystem analyzes a realistic SSF mockup under simu- 
lated orbital conditions, gathering sensory data indica- 
tive of potential problems. This inspection subsystem 
is the topic of this paper. The key technology items ad- 
dressed are: methods for automated visual inspection; 
the development of an Integrated Sensor End- Effector 
(ISEE) which encompasses vision, proximity, tempera- 
ture, and gas information to monitor the environment; 
and a high fidelity simulation of orbital inspection con- 
ditions. In this paper, each of these will be described 
as well as the issues which they successfully address. 

The paper is organized as follows: Section 2 dis- 
cusses automated visual inspection in detail, including 
the issues of ambient light and registration error com- 
pensation, as well as flaw and error models. Section 3 
describes the ISEE, and provides a detailed discussion 
of the use of proximity sensors for collision avoidance 
and surface following. Section 4 discusses the simulated 
conditions for the inspection operations, including a de- 
scription of the SSF truss mock-up and its temperature 
and gas anomalies, as well as a solar simulator which 
provides realistic orbital lighting conditions. Finally, 
Section 5 provides a summary and some conclusions 
drawn from this technology development research. 

2 Visual Inspection 

The approach adopted for on-orbit inspection of space 
platforms consists of locating and characterizing flaw- 
induced changes between an earlier reference image 
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and a new inspection image. In the absence of noise, 
viewpoint differences, lighting variations, and benign 
changes, the detection of significant new damage could 
be obtained by a process of simple differencing. How- 
ever, on-orbit use of robotic machine-vision to achieve 
this goal is constrained by a number of technical chal- 
lenges: 

• Imaging Repeatability. Subsequent scans of 
the space platform will not be able to achieve 
the same imaging view-point because of the lack 
of robot positioning repeatability and the expan- 
sion/ contraction of space platform structures. This 
can result in mis-registered reference and inspec- 
tion data sets, as well as previously occluded fea- 
tures being made visible and mistaken for new 
flaws. The presence of the flaw itself can com- 
plicate the recognition of the extent of the mis- 
registration. 

• Lighting Variation. In orbit, surface appearance 
can change drastically due to the variation in am- 
bient light (solar and earthlight) illumination in- 
duced by orbital motion. Power constraints on ar- 
tificial illuminators restrict the illumination tech- 
niques that can be adopted to compensate for this 
variability. Furthermore, the lack of atmospheric 
dispersion of the harsh solar light results in images 
having a large dynamic range with sharp shadows. 

• Flaw and Object Appearance. The surface 
flaws caused by micro-meteorite damage are very 
small 1 mm) [7] and must be detected on 
man-made objects with complex geometric shapes 
and constructed with specular materials. Benign 
changes such as the gradual reflectivity variation 
resulting from exposure to ultra-violet radiation 
and atomic-oxygen can mislead the inspection sys- 
tem. 

• System Constraints. Efficient computer pro- 
cessing is a must, given the computational lim- 
itations imposed by the need to use compact, 
light-weight, low-power, space- qualified comput- 
ers. Communication limitations in sending data 
back to Earth must also be considered in deciding 
on the partitioning of the image processing func- 
tions between the spacecraft and the ground. Data 
storage of the various reference images is less of a 
problem than would initially appear, thanks to the 
availability of space-qualified mass storage devices. 

• Motion Constraints. Robot motions can in- 
duce significant platform disturbances due to robot 


start/stop motions. If the disturbance is to be min- 
imized by performing all of the imaging from a con- 
tinuously moving sensor platform, then the result- 
ing problems of motion blur must be addressed. 

In this report the focus is mainly on the effects of am- 
bient light variability and image mis- registration, and 
the methods used to compensate for them. A brief dis- 
cussion on flaw-models and the quantification of the 
flaw detection performance is also presented. A detailed 
presentation may be found in reference [1]. 

2.1 Laboratory Imaging System 

The imager consists of an industrial color Charge Cou- 
pled Device (CCD) camera. With solar illumination 
at earth orbit at approximately 130000 lux, the to- 
tal illumination on a typical inspection scene area of 
0.1 m 2 over the duration of a single video field (1/60 s) 
is approximately 215 lumen * s. This is many times that 
which can be provided by a low-powered artificial light 
source, especially if it were a continuous illuminator. In- 
stead, artificial illumination is provided by an electronic 
strobe unit, with the laboratory unit providing an illu- 
mination of 1.3 lumen • s. When the strobe is used with 
the electronic shutter in the camera set to 1/10000 s, 
the total ambient solar illumination of the scene is only 
1.5 lumen • s, making it comparable to the strobe pro- 
vided illumination. Note that the total strobe illumi- 
nation remains unaffected by the electronic shutter ac- 
tivation because the strobe duration (« 20 //s) is still 
much shorter than the exposure duration. The over- 
all energy consumption for strobe lighting is also lower 
since the strobe is only used when the sensing platform 
traverses a new view-point. Further, the use of a short 
exposure time reduces the effects of motion-blur in de- 
grading the images. (As a practical note, since the lab- 
oratory ambient light simulator, described in Section 4, 
cannot achieve full solar intensity, the camera electronic 
shutter is operated at a somewhat larger setting. This 
effectively achieves the same ambient-to-artificial illu- 
mination ratio relevant to orbital operations.) 

The camera is operated with a unity gamma re- 
sponse. Any deviations from a linear response are com- 
pensated for in the digitizer. Linear response ensures 
that image intensity is proportional to scene radiance 
and allows linear operations (e.g. subtraction) on im- 
age fields to be correctly computed. This is required 
for the ambient light variability compensation meth- 
ods discussed in the next section. All imaging is per- 
formed using only the luminance signal of the video 
signal (quantized to 8 bits) with the color subcarrier 
information suppressed. 
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2.2 Ambient Light Compensation 

Ambient light subtraction uses two image data sets to 
obtain a compensated image. The first data set is illu- 
minated only with the ambient light and the second is 
illuminated with the ambient light as well as the arti- 
ficial illuminator. The information in the first data set 
is subtracted from that in the second to give a compen- 
sated image that appears as if it were taken with the 
artificial illuminator alone. In order for the subtraction 
results to be valid, the sensor response is required to 
be linear. There is, however, a reduction of the signal- 
to-noise (S/N) ratio since the subtraction process can 
nearly double the noise power in the data. Further, 
the utilization of the dynamic range of the camera is 
also reduced since the sensor cannot be allowed to sat- 
urate when both ambient and artificial illumination is 
utilized. The performance of ambient light subtraction 
is enhanced when the artificial illuminator provides en- 
ergy comparable to (or more than) the ambient light. 
As discussed earlier, the electronic shuttering mecha- 
nism achieves this. Note that strobe illumination is es- 
sential here for operating with a moving imaging plat- 
form since continuous illuminators, even if low power 
and high-intensity, would take a finite amount of time 
to ramp up to the desired intensity level. This would re- 
quire the imaging platform to be stationary during the 
taking of the two image data sets necessary to achieve 
compensation. 

An additional problem is that in a strobe illuminated 
image only one of the 2 : 1 interleaved image fields (say 
the odd-field) is lit by the strobe, while both fields are 
ambient lit. An estimate of the ambient light compo- 
nent in the odd-field is generated from an average of 
the ambient light data in the even-field immediately 
above and below each odd-field image scan line. A com- 
pensated image is generated by intra-frame subtraction , 
wherein this ambient light estimate is subtracted from 
the odd-field data. 

This process does suffer from some disadvantages, 
namely a halving of the vertical resolution in the com- 
pensated image and the possibility of interpolation er- 
rors when estimating the ambient-lit component of the 
image. As expected, if the same ambient light is used in 
the reference and inspection images, then the interpola- 
tion errors are identical and cancel when performing the 
subsequent image comparisons for flaw detection. Any 
non-zero change can then be attributed to the presence 
of a new flaw. 

However, interpolation errors are of consequence 
when the ambient light changes, and lead to an in- 
creased probability of false errors during the flaw de- 
tection process. For two special cases which correspond 
to limiting cases typically encountered in real applica- 


tions, the deleterious effects of the interpolation error 
is manageable. The first case corresponds to when the 
ambient light illumination of the surface for both the 
reference and inspection images has low spatial varia- 
tion and the underlying reflectivity of the surface un- 
dergoes a large change. Here analysis shows that the 
significant errors only happen in regions where the re- 
flectivity changes are large, which are precisely the same 
regions where mis-registration errors due to sensor-to- 
platform positioning errors can be expected to be of 
greater significance. 

The second case occurs with ambient light discon- 
tinuities at shadow boundaries. If the transition from 
light to dark in the “pen-umbra” region of the shadow 
is very sharp, then the estimate generated by interpo- 
lating the even-field data will be incorrect. If, however, 
the transition occurs over a spatial extent of more than 
a couple of pixels, then the interpolation process will 
be able to accurately estimate the ambient light in the 
middle of the shadow boundary region. The extent of 
the pen-umbra region is a function of the distance from 
the surface to the object casting the shadow. If the ob- 
ject is close to the surface then the transition is sharp, 
and conversely if it is far away from the surface the 
transition is more smooth. Assume that a pen-umbra 
region greater than 2 pixels is of sufficient spatial extent 
to permit the interpolation to be reasonably accurate. 
An estimate of the corresponding object distance that 
would generate such a shadow can be easily obtained 
from simple geometrical arguments. For a typical field- 
of-view and imaging standoff-distance, a shadow transi- 
tion region of 2 pixels corresponds to to a surface spatial 
extent of about 1 mm. Noting that the sun subtends 
approximately 0.01 radians, and that the shadow pen- 
umbra must necessarily subtend the same angle, gives 
the corresponding object distance as being 0.1 m. Thus 
sharp shadows will only be cast by objects closer than 
0.1 m to the surface. Even for such sharp shadows, 
the situation is ameliorated by the fact that the result- 
ing interpolation errors are localized to a region along 
the shadow boundary that has a very narrow width. If 
the flaw being detected has a spatial extent larger than 
this width, then the resulting errors during flaw detec- 
tion are reduced. This issue is discussed further in the 
Section on flaw models (Section 2.4). 

2.3 Registration Error 

Registration errors are induced by the lack of repeata- 
bility in the viewpoints at which images are taken for 
the reference and inspection images. These viewpoint 
discrepancies arise due to the inherent accuracy limi- 
tations of moving camera platforms. In the laboratory 
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environment, i.e., fixed targets and industrial arms with 
good repeatability, the inaccuracy translates to no more 
that one to two pixels when images are taken from 
relatively short distances of less than 0.7 m. In the 
space environment, larger repeatability problems are to 
be expected due to arm flexibility and object location 
changes due to thermal expansion and structural flexi- 
bility. 

With this mis-registration, the comparison of com- 
pensated images by performing a simple subtraction of 
the compensated reference and inspection images re- 
sults in a number of “false edges” in the differenced 
image. The magnitude of registration error depends 
on both the directional gradient of the gray-level image 
with respect to the camera platform motion parame- 
ters, as well as the occlusions at each imaged point. 
Here, only the directional gradient with respect to lat- 
eral and horizontal motion of the camera platform are 
considered, since these are expected to dominate for 
this inspection application. Occlusion induced errors 
are also not considered, even though their effects could 
be significant near any sharp depth deviations in the 
image. 

A Gauss-Newton iterative method is used to per- 
form re ference-to- inspection image registration prior to 
making the comparison. The residual sum of squares 
between the actual and an estimated picture is used as 
an evaluation function to indicate the degree of match 
between the inspection data and a transformed refer- 
ence image. The objective is to find a suitable trans- 
formation of the reference image so that the residual is 
close to zero. The Gauss-Newton algorithm solves this 
nonlinear least-squares problem via an iterative solu- 
tion method and exploits the special structure of the 
gradient and Hessian matrix of the evaluation function 
[3]. The iteration process is continued until the least- 
squares residue drops below an acceptable threshold, 
at which point the estimate can be considered to be 
registered with the data. Note that terms involving 
the Jacobian matrix in the case of pure translational 
mis-registrations can be pre-computed resulting in sig- 
nificant run-time computational savings. Nevertheless, 
residual mis-registration is still possible because of early 
termination of iterative registration correction necessi- 
tated by real-time deadline processing constraints. 

2.4 Flaw and Error Models 

The process used to detect a flaw is intimately linked to 
the corresponding model of the flaw. Flaw models must 
provide a reasonable approximation to the physical ap- 
pearance of the flaw while not being overly complex to 
preclude implementation of the associated flaw detec- 


tion algorithms on a real-time system. Two types of 
flaw models are presented and the corresponding flaw 
detection processes are characterized. 

A single-pixel flaw model treats each individual pixel 
independently of other pixels when it comes to flaw de- 
termination. A flaw is assumed to be present at a pixel 
if the surface intensity at that pixel in the inspection 
image differs from the surface intensity in the corre- 
sponding reference image pixel by a value greater than 
a characteristic flaw strength. The characteristic flaw 
strength is a function of the flaw type and can be deter- 
mined by examining images of known and/or calibrated 
flaws. 

In a multi-pixel flaw model a flaw is assumed to be 
present at a pixel if it occupies a certain minimal spatial 
extent. More precisely, consider for both the inspection 
and reference images, the corresponding surface inten- 
sity vectors each comprised of the intensity values in 
a spatially connected region around that pixel. A flaw 
is assumed to be present at the pixel if these vectors 
differ from each other by greater than a flaw strength 
vector. Once again, characteristic flaw strength vectors 
are a function of the flaw type and can be determined 
by examining images of known and/or calibrated flaws. 

Two special cases may be considered depending on 
the nature of the flaw model vector. The first of these, 
is the uniform flaw model which takes each component 
of the flaw strength vector to be equal. This model 
is suitable in cases where the flaw has a uniform ap- 
pearance across the entire neighborhood (e.g. a spot of 
paint on a surface). The second is the peak/adjacent 
flaw model which takes all but one component of the 
flaw vector to be constant with the exception being one 
single component which has a higher absolute magni- 
tude value than the others. The second type is suitable 
where the flaw has a strong peak value surrounded by 
adjacent pixels with smaller but uniform values. This 
provides a crude approximation to the flaw morphology 
of micro-meteoroid impact craters where the center of 
the crater is darker than the rest. 

Given the definition of a flaw, the null decision hy- 
pothesis Ho assumes that there is no flaw. The flaw de- 
cision hypothesis Hi assumes that a flaw is superposed 
onto the reference image. In order to determine if a 
flaw is present, the log likelihood ratio [10] is checked 
to see if it exceeds the test threshold. 

Working out the details in the single-pixel case indi- 
cates that, as expected, given compensated images cor- 
responding to reference and inspection images, the flaw 
detection can be performed by locating flaws at all pixel 
locations where the differenced image exceeds a pre- 
determined threshold. For the multi-pixel model, the 
flaw detection process involves taking weighted sums of 
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the differenced image in a suitable window and com- 
paring these sums to a pre-determined threshold. A 
sub-optimal version of the detection test can be imple- 
mented using morphological erosion operations. 

With the appropriate model for the flaw, the theoret- 
ical flaw detection performance can be analyzed. The 
performance is dependent on the distribution of the flaw 
detection signal under the two competing hypothesis: 
the Null Hypothesis Ho and the Flaw Hypothesis H\. 
If these distributions do not overlap, then it is possible 
to pick a threshold parameter for the detection pro- 
cess such that all flaws that occur are detected, and at 
the same time no false- alarms are generated. However, 
the distributions of the signal under both hypothesis 
do overlap because of the nature of the noise in the 
imaging process, and as a consequence for any threshold 
parameter, there will always be a possibility of missing 
a flaw and of falsely identifying a flaw. The selection 
of the threshold affects the performance of the system 
and is a function of the characteristic flaw strength and 
the noise levels in the system. Too high a threshold 
will decrease the probability of detection Pd , while too 
low a threshold will increase the probability of a false 
alarm P F . This aspect of the performance is captured 
by providing parametric plots of the Pd versus P F for 
various cases. These plots are known as Receiver Op- 
erator Characteristics (ROC’s) from their earlier use in 
radar target detection. A detail analysis of performance 
has been conducted using these concepts and presented 
elsewhere [1], 

Errors in mis-registration correction and ambient 
light compensation can be interpreted as increasing 
the noise in the image leading to lower detection per- 
formance. Residual mis-registration errors induce a 
change in intensity which can be confused with a flaw. 
Only translational mis-registration effects are consid- 
ered here since any mis-registration effects arising due 
to small angular motion in the image plane may be lo- 
cally approximated as a translational mis-registration. 
An analysis shows that the intensity difference at a pixel 
due to mis-registration may be considered as an addi- 
tional noise term that adds on to the more typical ran- 
dom noise components present in an image. The pres- 
ence of mis-registration increases the threshold which 
must be exceeded before a difference value is considered 
to be a flaw, and consequently reduces the possible per- 
formance. In a similar way, interpolation errors during 
ambient light compensation can also be interpreted as 
a noise term distributed over the image. If these noise 
effects are localized then they have less of an impact on 
the multi-pixel flaw model likelihood-ratio test than on 
the single-pixel case. This is because of the averaging 
inherent in determining the likelihood ratio test in the 



Figure 1: Large residuals are detected at Raw loca- 
tions. 


multi-pixel case. 

A number of tests under different lighting conditions 
have been performed to test the flaw detection algo- 
rithms. Flaws are simulated by a random dot pattern 
of a given pixel size distributed on the surface of a test 
object. Figure 1 shows the final differenced image after 
mis-registration correction. 

2*5 Visual Inspection Summary 

The key conclusions are summarized: 

• Image differencing appears to be a viable approach 
for flaw detection with the use of ambient light 
compensation methods and iterative registration 
algorithms to overcome the problems of variable 
lighting and image mis-registration. 

• The Gauss-Newton algorithm has been shown to be 
effective in performing mis-registration correction 
with large (« 10 pixel) registration errors. 

• Issues relevant to a flaw- detection theory have been 
presented and applied to test cases in the labora- 
tory. The quantitative tools developed allow an 
explicit tradeoff between detection probability and 
the false-error probability. Depending on the flaw 
model and noise parameters, detection thresholds 
can be chosen to achieve a given level of perfor- 
mance. 

Areas of further work and necessary improvements are 
identified: 
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• Active Inspection Strategies need to be developed 
to improve data collection upon preliminary detec- 
tion of a potential flaw. The additional data would 
be used to improve detection performance and 
could involve commanding additional sensor plat- 
form motions to improve lighting and viewing an- 
gles; and selection of different illuminator/camera 
combinations to get more data. 

• The information in the ambient lit image needs 
to be exploited and used to supplement the im- 
age information in the compensated image. In the 
ideal case, the strobe light should be used only to 
“probe” or supplement the ambient lit image for 
additional information. 

• Multiple imaging with different electronic shutter 
settings needs to be investigated in order to im- 
prove the dynamic range in both bright and dark 
regions of image. 

• Flaw morphology data needs to be captured 
by supplementing the imaging sensor with a 
depth/profile sensor. 

• Occlusion data needs to be generated at each vista 
point to allow the anticipation of previously oc- 
cluded portions of the scene being mistaken for 
flaws. This might require data from an additional 
camera or from an additional image taken near 
each vista point. 

3 Integrated Sensor End-Effec- 
tor 

While visual inspection is the primary means of flaw 
detection, it is only one of the modes available. There 
are some anomalies, such as errant temperatures and 
gas leaks, which are not directly detectable by visual 
information. Therefore, a compact Integrated Sensor 
End-Effector (ISEE) has been developed to house not 
only the cameras and lights, but a suite of other sensors. 
Figure 2 shows the recently constructed device, where 
the labeled components are: 

A Two intensity feedback controlled halogen lamps. 

B Two fast pulse strobes flashes. 

C A parallel jaw gripper. 

D Two color cameras calibrated for stereo viewing. 

E Two infrared triangulation proximity sensors. 

F A six DOF force/torque sensor. 
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Figure 2: A front view of the ISEE. The lettered com- 
ponents are a described in the text . 

G An optical pyrometer with a laser sighting beam. 

H A Metal Oxide Semiconductor (MOS) gas/vapor 
sensor. 

Proximity sensing is achieved with two infra-red tri- 
angulation sensors, sensitive to approximately 0.75 m. 
The distance measurements are used for collision avoid- 
ance, surface contour following, and surface contour 
measuring. Temperature sensing is achieved with an 
infra-red optical pyrometer (8-12 micron wavelength), 
sensitive to temperatures from 0 to 1000°F. Gas sens- 
ing is achieved with a multi-gas MOS type sensor which 
changes resistance as a vapor is absorbed. (While it 
may be possible to employ this gas sensing technology 
in orbit, we recognize the superiority of using a compact 
mass spectrometer in the ambient vacuum of space.) 

The controlled lights are maintained at a known il- 
lumination level by a optical transistor feedback cir- 
cuit. This makes the illumination independent of cur- 
rent fluctuations and bulb age, and makes precise mea- 
surement and camera characterization possible. This 
lighting is augmented by extremely compact and fast 
pulse strobes. The strobes provide short duration light- 
ing of intensity on the order of the Sun but only for 
short, energy saving, single camera frame, bursts. Since 
the flashes are mounted on the outside surface of the 
movable parallel jaws of the gripper, the flash illumina- 
tion angle may be varied as desired. 

All components are commercially available, and have 
been physically and electrically integrated into the corn- 
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proximity 


Figure 3: Experimental data showing the filtered prox- 
imity measurements from the two proximity sensors as 
a function of the arm position. The environment sur- 
face was at about -0.81 m. 

pact ISEE end-effector, with a resultant mass of ap- 
proximately 3.5 kg. The force and proximity sensors, 
as well as the gripper, are not directly used for inspec- 
tion. Instead, they aid in the control of the robot arm, 
and therefore, the end-effector. In particular, the prox- 
imity sensors can be used for collision prevention and 
surface tracking. The development of these capabilities 
is discussed next. 

3.1 Proximity Sensors for Inspection 
Operations 

A demonstration of the utility of proximity sensor en- 
vironmental position determination for robot collision 
avoidance has been performed in a real-time implemen- 
tation. For these tests, two IDEC/Izumi SAID triangu- 
lating range sensor were used [11, 12]. Since the sensors 
have a minimum sensing distance, they were recessed 
with respect to a parallel jaw gripper which has a length 
of 11 cm. The sensor values were read through an A/D 
board by a 68040 processor (VME bus architecture) at 
a sampling rate of 44 Hz, and the data was digitally 
low-pass filtered for noise reduction. Figure 3 shows 
the filtered readings from the two proximity sensors as 
a function of the robot end-effector position. The re- 
sponse is fairly linear and consistent between the two 
sensors. 

To use the proximity sensor readings for control of 
the manipulator, the velocity x ps in the block diagram 
of Figure 4 was commanded as a function of the sensed 
distance. Two different functions were used: collision 
avoidance and distance servoing . Figure 5 shows these 
two functions, which are identical except for the dashed 
segments of the servoing function in regions D and 
E. The piece- wise continuous formulation was chosen 
mainly for simplicity in implementation and ease of 
modification. The value of V js is the maximum velocity 


Figure 5: The piece-wise continuous functions of the 
commanded velocity x ps as a function of sensed prox- 
imity. The collision avoidance and distance servoing 
functions are identical except that the latter has posi- 
tive values indicated by the long dashed line. See the 
text for a fuJJ description . 

that can be commanded from the joysticks. Operating 
region C provides a collision avoidance velocity com- 
mand that can not be overridden with a large positive 
velocity command, ij S , from the joysticks. Operating 
region B allows for quick retreat of the arm if environ- 
mental surface protrusions should come into view from 
the periphery as the the arm is moved tangential to the 
Surface. (It is desirable to restrict the slope and abso- 
lute magnitude of the function in this region because of 
the low sampling rate employed. For instance, had an 
asymptotic function been employed, there would exist 
the chance of a very large or rapidly changing velocity 
command near the asymptote position.) Finally, re- 
gion A will typically never be entered since the sensors 
are recessed, and the sensor is incapable of determining 
distances at this range. 


Regions D and E have non-zero values only for dis- 
tance servoing (the long dash lines in Figure 5). In D, 
the slope is matched to region C, to provide equal accel- 
eration to the servo point between C and D. The peak 
value of the distance servoing velocity is restricted, to 
allow negative joystick commands to overcome it and 
‘puli’ the arm away from the surface. Region E is pro- 
vided to make the function continuous. In region F , the 
sensor can detect distance, but the commanded velocity 
is zero. Outside of F the sensor is out of range. 

Figures 6 and 7 show the values of x ps commanded 
by the avoidance and servo functions in the real-time 
implementation. For these measurements was not 
added to i r , and a simple linear trajectory away from 
the environmental surface was used for it g . 
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Figure 4: Block diagram of the control system used for the initial tests of proximity sensor collision avoidance. 



Figure 6: Experimental data showing the commanded 
repulsion velocity as a function of measured proximity 
to the environment. 

4 Orbiter and Sunlight Simula- 
tion 

To demonstrate the capabilities of the inspection sys- 
tem, a one-third scale mock-up of the Space Station 
Freedom truss has been created. Figure 8 illustrates 
the mock-up and its components: 

A Electrical Orbital Replacement Unit (ORU) which 
opens to the left on a hinge to reveal electrical con- 
nectors and a Cold Plate. 

B Tank ORU. 

C Solar Panel. 

D Tank and Tubing ORU. 

E Simulated hot and cold spots. 

F Simulated micrometeor impacts and gas leaks. 

The simulated hot and cold spots on the electrical ORU 
are created using Peltier effect heat pump modules 



Figure 7: Experimental data showing the commanded 
distance servo velocity as a function of measured prox- 
imity to the environment. 


mounted on the inside of the aluminum surface. Since 
the aluminum has a low emissivity, the outside surface 
is covered with a circle of Black Kapton to enable the 
surface temperature to be correctly measured by the 
optical pyrometer. In the future, the surface temper- 
ature may be measured directly by touching it with a 
thermocouple, eliminating the need for the Kapton. 

To introduce a degree of randomness into the inspec- 
tion process, only two of the Peltier modules are turned 
on at any time, and the selected direction of electrical 
current determines if the surface becomes hot or cold. 
A similar selection is available from amongst the three 
possible gas “leaks”. Each leak uses compressed air to 
spray a fine mist of household ammonia (to simulate 
hydrazine) from a small hole on the Tank and Tubintr 
ORU. 8 

Also, random defects may be introduced into the 
truss mockup through three simple methods. First, 
screws throughout the truss can be randomly removed 
to indicate structural defects. Second, small pieces of 
black tape on pen markings can be placed throughout 
to simulate micrometeorite impact sites. Third, entire 
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Figure 8: The mock-up of the SSF truss. The lettered components are described in the text. 


components, such as tanks, can be replaced with defec- 
tive versions. 

Finally, to simulate the space environment around 
the truss, the mock-up and the inspection robot have 
been placed in a room darkened by black curtains [4]. 
The operator can view the mockup and inspection op- 
erations from one of three stereo camera views or from 
a window of the SSF cupola mock-up, in which the in- 
spection station is situated. Simulated sunlight is pro- 
vided by the Solar Illumination Simulator, discussed 
next. 


4.1 Robotic Lighting Control for Solar 
Illumination Simulation 

Traditional solar simulators are designed for thermal 
tests of actual spacecraft [5]. To accomplish this, they 
utilize large vacuum chambers to house the spacecraft, 
and collimated lighting from arrays of xenon arc lamps. 
Brightness up to an order of magnitude greater than 
solar intensity is possible. To test the effects of changing 
lighting direction, the entire spacecraft is rotated while 
the illumination remains constant. While this approach 
is necessary for pre-flight spacecraft testing, it is simply 
not practical for robotic system prototype development. 

Alternatively, we have developed a small scale sim- 
ulator which effectively mimics the relative motion of 
the Sun in the sky, while still providing realistically 
scaled illumination levels [13]. Figure 9 is a photograph 
of the simulator, a 1500 Watt arc lamp mounted on a 
four degrees-of- freedom, computer controlled platform. 
Its ability to pan/tilt/translate, as well as modify the 
beam shape, enable the illumination angle of the scene 
to be varied at rates equal to those experienced in low 
Earth orbit, and maintain a constant illumination flux 
just as the Sun provides. While the simulated solar il- 
lumination is only 1.5% that of true orbital sunlight, 
Section 2.1 has previously described the compensating 
adjustments of controlled lighting position, strobe light- 



Figure 9: A photograph of the solar illumination sim- 
ulation system’s robotic hardware. 


ing pulses, and camera exposure times, provided by the 
inspection system [1]. Therefore, the lighting conditions 
are a realistic test for machine inspection algorithms 
and human operators. 

Figure 10 shows the solar illumination simulator as a 
five DOF system, which is represented by its state vec- 
tor of configuration variables, 0 = (p, 0, tp } A, 7 ), where: 


p, Spherical coordinates from the lamp cen- 

ter to the projected spot center. 

A TVavel of lamp from its origin frame. 

7 Lamp focus parameter indicating position 
of bulb carriage on internal lead screw. 


These parameters have the following ranges: 


139 





Figure 10: The configuration and task coordinates for the solar illumination simulator system. 
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The corresponding task state vector, x — (“'A,s,Z), 
is composed of the following variables which are also 
shown in Figure 10: 

“'A Cartesian vector from world frame to cen- 
ter of projected spot. 

s Beam angle from the lamp frame n axis. 

T Light intensity at the center of the spot 
on the environment. 

The task vector is obtained from the configuration vec- 
tor through the forward kinematics: x = F(0). 

Finally, it is important to note that although the 
kinematics has five DOF, only four are actuated. In the 
configuration space, the unactuated and unmeasured 
DOF is the radial distance from the lamp to the surface, 
p. It’s value is calculated from the user specified world 
coordinates, ^A. The controller is open-loop for this 
variable since no real-time measurement of p is possible. 

In the task space, the unactuated and unmeasured 
DOF is the light intensity at the surface. Maintenance 
of the intensity is performed open-loop based on the 
calculated value of p and an optics model which has 
been experimentally verified [13]. 


5 Conclusions 

This paper has presented the details of some of the 
technology developed for telerobotic inspection of space 
platforms such as SSF. Primary amongst the inspection 
technologies has been visual inspection using computer 
processing of images from robotically controlled cam- 
eras. The processing provides ambient light compen- 
sation, registration correction, and automatic flaw de- 
tection based on the described flaw models. Secondary 
inspection and other sensory data are provided by gas, 
temperature, proximity, and force sensors integrated 
into the compact ISEE end-effector. This device has 
been described and the proximity sensor based control 
of collision avoidance and surface following has been 
highlighted. Finally, a complete description has been 
given for the simulated orbiter defects and the space 
environment lighting. This simulation environment has 
allowed more rigorous testing of the developed inspec- 
tion devices and methods. 
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